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Introduction 


The objective of this paper is to provide an outline for an approach to 


cause-effect analysis for developing observations in systems audits. 


By using the example of the General Purpose Accommodation Program 
of DPW, this paper demonstrates how cause-effect relationships could be 
identified. 


The tool used is a matrix, which we found to be convenient for the 
purposes of this analysis. The focus is on the approach rather than on the logie as 


the logic is always a matter of professional, and intellectual, judgement. 
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Construction and Use of a Matrix 


Figures 1 and 2 show the structure of the matrix. Matters of 
significance to Parliament, in terms of economy, efficiency, effectiveness and 
reporting, are listed across the top. On the vertical column, we would list all the 
key controls we expect to see in the system. These may relate either to the 
management process or to the decision process. The relationships between 
matters of significance and management or decision processes are identified at 
the intersection of these columns. These relationships are identified by asking the 


question: 
What could happen if a certain control is weak or non-existent? 


The matrix is started at the same time as we chart the system and 
develop logic models. Although the matrix helps in building the logie models, it 
cannot be completed without them since they provide the vertical axis 


information. 


Conversely, the matrix helps in defining matters of significance. It 
achieves this by forcing the auditor to decide on matters of significance at an 


early stage of the audit. 


While the example of acquisition of capital assets is used, the 
approach is also suitable, subject to appropriate modifications, to other situations. 
For example, the models could be applied to identifying waste in an operation 


processing application caused by: 


2 poor communication with client groups; 
= poorly designed forms; or 


= inadequately trained public servants. 


To illustrate the use of the matrix, let us assume that the auditor of 


DPW has the following hypotheses: 
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Management/ 
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Controls 


The Structure of a Matrix 


e Purposes 

e Objectives 

e Goals 

e Effects 

e Consequences 

e End 

e Economy, Efficiency, Effectiveness 
e Objectives of Control 

e Matters of Significance 


e Targets 


e 
k 


(a) 


(b) 


(ce) 


(d) 


structure of the matrix is shown in the general model (Figure 2). The 


management control functions are taken from the logic model (Figure 3) and 


Resource utilization is not economical. 


- excessive cost of asset; 


= excessive cost of operations and maintenance. 


Organizational products/services are not effective. 


= user needs are not met; 


= overestimated revenue. 


Inefficiencies in resource utilization. 


- excessive cost of asset; 


S excessive cost of operations and maintenance. 


Irregularity 


= unauthorized expenditures; 


- other projects may be adversely affected; 


= inadequate reporting to Parliament. 


These hypotheses are listed in the matrix as shown in Figure 1. The 


listed in the matrix. 


our examination. What we are actually saying is that the weakness or absence of 


option definition and analysis control function will likely lead, albeit indirectly, to 


By matching controls with possible effects, we can narrow the focus of 


any or all of the following: 
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(a) Actual asset cost will be excessive. 
(b) Operations and maintenance will be excessive. 
(ec) Revenue will be lower than acceptable level. 


(d) User needs will not be met. 


In addition, these potential consequences add up to make this control 
point a key one that merits our attention. And so on for other control- 


consequence relationships. 


We shall now concentrate on the economy issue. In particular, we 
shall examine the possible excessive cost of acquiring the asset. We want to find 
out which management control function permits this adverse effect. Is it poor 
forecasting of demand? Is it weak option definition and analysis? Is the Treasury 
Board process to blame? Or is the phase of design and construction at the root of 
the problem? Of course it is possible that a combination of the above, or indeed 
matters beyond the management process, are causes of the suspected adverse 
effect. 


Past experience shows that diseconomy in capital acquisitions could be 
the result of a number of things. To be on the safe side, we will look at all key 


controls shown by the matrix to have an impact on this economy issue. 
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Figure 3 
REAL ESTATE OPERATION, COMMUNICATION LINKS 
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Specifying Audit Criteria 


x Let us say the Department could reasonably be expected to acquire 
capital assets at economical costs if the following key controls or systems exist 


and function adequately: 


Demand Forecast}; 
Option Definition and Analysis; 


Treasury Board Process; 


Design and Construction. 


However, we need to identify audit criteria for each of these key 
systems or control functions. The audit criteria will be used as standards for 


evaluating managerial actions. 
We state these criteria by saying: There should exist the following: 
(a) Forecast of Demand 
- reliable forecast of demand; 
(b) Option Definition and Analysis 
= accurate data; 
= life-cycle costing; 
= accurate cost of financing calculation; 
= revenue estimates; 
~ proper analysis of alternatives; 
= ete.3 
(ec) Treasury Board Process 
= requests for funds to be supported by accurate and complete 


documentation; 


~ ete.; 
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(d) Design and Construction 


= competent architects; 


2 appropriate design changes control; 


= adequate project cost control; 


= ete. 


Having done his or her home work and narrowed the focus, the auditor 


begins to look deeper into those four control functions. 


Result of Preliminary Survey 


The auditor should be able to provide a preliminary assessment of the 


system and key controls. Such an evaluation could be presented in the format of 


Potential Finding Sheets (Exhibits 1, 2, 3 and 4). They represent tentative 


conclusions concerning each key control function. Each sheet has the following 


components: 


Component 


Title 


Hypothesis 


Criteria 


Condition 


Effect 


Cause 


Recommendation 


Significance 
To identify control function under 
examination. 
Tentative conclusion 


Standard for evaluating managerial 
actions. 


What was found through preliminary 
survey or organization and systems. 


What could go wrong - matters of 
significance 


To enable auditor to make con- 
structive recommendations 


For constructive auditing 


Source/Reference 


Matrix 


Matrix 
OAG 


Logic 

System descriptions 
Decision process 
descriptions 


Auditor's interpretation 
of AG mandate 
Matrix to a limited extent 


Auditor's judgement and 
experience 

Speculation 

Preliminary Survey 


Auditor's judgement 
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It may be helpful to be more specific on the sources/references for the 


potential causes listed above by including: 


- knowledge of the business, client and past history; 
= observation; 

= enquiry and confirmation; 

- inspection of documents; 

= computations; and 


= analysis. 


12. 
Exhibit 1 
POTENTIAL FINDING NO. 1 


Demand Forecast 


Hypothesis: Cost of asset is likely to be excessive. When 


unable to forecast demand reliably, the 
Department tends to provide bigger and costlier 


buildings. 


Criteria: There should be a reliable forecast of demand. 


Condition: Department cannot possibly control key 
factors in accommodation demand. 
No clear definition of responsibility and 
accountability. 
Almost impossible to forecast reliably the 
government need for office accommodation. 
Lack of clearly described, written 
objectives noted in past procedures. 
Ete. 


Effect: Possible excessive cost of asset. 


(Not identified). 


Recommendation: (Not yet made). 
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Exhibit 2 
POTENTIAL FINDING NO. 2 


Title: Option Definition and Analysis. 
Hypothesis: Cost of asset excessive. 


Criteria: There should be a comprehensive option definition and 


analysis including: 


accurate data; 

life-cycle costing; 

accurate cost of financing calculation; 
precise revenue estimates; 

proper analysis of alternatives; 


ete. 


Condition: Quality of information used as input into the 


computer-based investment model is poor. 
Life-cycle costing omitted. 

Cost of financing calculations inaccurate. 
Revenue estimates imprecise. 


Alternatives not properly analysed. 


Possible excessive cost of asset. 


Advocacy and uncritical acceptance of costly methods of 


acquisition (lack of objectivity). 


Recommendation: (Not made yet - but could be made in management 


letter - eliminate condition and comply with criteria). 
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Title: 


Hypothesis: 


Criteria: 


Condition: 


Effect: 


Cause: 


Recommendation: 
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Exhibit 3 
POTENTIAL FINDING NO. 3 


Treasury Board Process. 


Non-compliance with TB Process may result in excessive 


cost of asset. 


These should be adequate control by TB over DPW's 


expenditures including: 


requests for funds should be supported by accurate 
and complete documentation; 


ete. 


Life-cycle costing not presented to TB. | 


Chosen alternatives presented as the most 
economical. 

Inflation not accurately anticipated. 

No detailed examination by TB. 

Ete. 


Possible excessive cost of asset. 


ADM's wish to present his plans to TB as the best ones 


(human nature: wish to be right). 


(Not made yet - but could be made in management 


letter - eliminate condition and comply with criteria.) 
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Title: 


Hypothesis: 


Criteria: 


Condition: 


Effect: 


Recommendation: 


Exhibit 4 
POTENTIAL FINDING NO. 4 


Design and Construction. 


Design changes and weak project cost control will likely 


lead to excessive cost of asset. 


There should be: 


competent architects; 

no substantial design changes without approval by 
TB; 

adequate project cost control; 


ete. 


Substantial design changes are allowed by TB. 


Excessive cost of asset. 


(Not identified yet). 


(Not made yet). 
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Preparing Audit Plan 


< The results of the preliminary evaluation indicate weaknesses in all 


four controls. 


The objective for the audit plan should now be clear: To illustrate to 
the auditee's management the extent of dollar costs that could occur as a result 
of deficiencies in the management process. By illustrating, at the survey phase, 
the possible cost effect resulting from weaknesses in the management process, 


our potential recommendations will be difficult to refute. 

How to establish satisfactory audit evidence of the non-existence of 
significant controls in the management process is, among other things, a sampling 
problem familiar to auditors and will not be discussed here. Let us assume that: 

(a) The auditor planned to examine: 

= a project in the planning stage; 

= a partially constructed building; 

a building that has just been completed; 

x a building that has been completed for a number of years. 

(b) The auditor's resources were: 

Staffing: 

= a CA; 

- an MBA; 

- an architect. 


Advisers: 


= quality assurance/challenge edit; 
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= lawyer; 


- commercial lessor. 


(c) The results of examining one project (C.D. Howe Building) are now 


available. 
Substantive Test Results 


The results of substantive tests of C.D. Howe Building showed the 


following adverse effects:!- 


= Cost of project reached $102 million compared to planned and 
approved cost of $57 million. 
= The method used to acquire the building cost approximately $12 


million more than an alternative method of acquisition. 
Cause-Effect Analysis 


With reference to our tentative conclusions and all other further 
evidence, we have a better understanding of the causal complex (Figure 4). For 
example, we find out that design changes are the personal choice of the project 
architect. We also recognize that management process did not allow for reliable 
forecasting of inflation, and that inflation was actually high during the 


construction period. 


We obtained sufficient appropriate evidence that enabled us to 
eliminate other potential causes for excessive cost, such as strikes, collapsing 
components of the building during construction due to poor design, poor materials 


or poor construction work, and so on2, 


In fact, we may break down causes into direct and indirect as follows: 


- See 1978 AG Report p. 52 


Z. Demonstrating how the logic was worked out and evidence obtained is 
beyond the scope of this paper. 
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e Advocae 
Direct y 


e Inflation 
e Design changes 
Causes 
e Weak option definition and 
analysis 
: e TB process not followed 
Indirect e Inflation forecast unreliable 


We also note that most indirect causes are related to the management 


process. We now have significant data to be inserted in the matrix (Exhibit 5). 


When we think of cause we think of the origin, source, prime mover or 
reason that leads to or produces the effect. In other words, it should answer the 


question "Why did it happen?" 


We may now attribute causes to effects although a separation may not 
always be possible. For example, we may be able to say that the management 
system and practices allowed the ADM to choose an outside contractor that, in 
turn, led to the $12 million in extra costs. Similarly, we may be able to say that 
the state of the economy led to inflation which, in turn, led to increased costs 
from planned. Finally, the choice of the project architect, compiled with allowing 


substantial design changes, also caused increased costs from planned. 


The distinction between direct causes and indirect causes is an 
important concept. Indirect causes are the underlying circumstances that permit 


the direct cause to operate. 


Often we are more concerned with the underlying circumstances than 
with direct causes. The reason for this is practical. Direct causes are frequently 
laws of nature (accidents, history, human nature, etc.). Most interferences with 
laws of nature are ineffective or prohibitively costly. However, we are able to 
minimize the chances for the direct causes to operate by eliminating the 


underlying circumstances. For example, the uncritical advocacy of costly 
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projects by a senior public servant, for which audit evidence was obtained, has 
roots in understanding human behaviour. Rather than attempting to change 
human behaviour (the direct cause), we could prevent the unwanted effect by 
insisting that alternative methods of asset acquisition be fully examined. Very 
few public servants will actually choose the more expensive alternative if they 
have full information. We could also reeommend that an independent party 
examine the proposals with a critical mind, having obtained full information (e.g., 


Treasury Board). 


Figure 4 
FIGURE 4: CAUSAL COMPLEX FOR EXCESSIVE COST OF PROJECT 
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Exhibit 5 


Actual Adverse Effects 


= ee 


WEAK 
CONTROL 
FUNCTIONS 


Unauthorized 
Expenditures 


Excessive 
Cost-Asset 


Demand 
Forecast 


$12 million 
more than an 
alternative 

method of 
acquisition 


Option 
Definition 
and 
Analysis 


$12 million 
more than an 
alternative 

method 


e $102 million 
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$57 
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Process 


e $102 million v. 
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Developing Observations and Recommendations 


2 The audit finding which emerges may be presented as in Exhibit 6. 


The information sources are detailed below. 


Although knowing the causes facilitates our task, our 
recommendations are more effective if we concentrate on eliminating the 


conditions, since they relate more closely to management processes and systems. 


Audit Observation Component Source 
Title Matrix 
Criteria OAG 
Condition e Logie models 
e System descriptions 
e Decision process description 
e Result of examination 
Effect Result of examination 
Cause e Result of examination 


e Judgement and experience 


Recommendation Judgement and experience 
Cause-effect relationship Judgement and experience 
Condition-effect relationship Matrix 


Conclusion 


To prove conclusively that an adverse effect is attributable to a 
particular weakness in a management control function requires extensive, costly 
and rigorous testing. Even then the results will most likely show that other 
management controls are involved. Indeed, matters beyond the control of 


management often cause such adverse effects to happen. 


Gefen * astiibro) 
enon mal sy * “es. 
anitghorel wesc foleioe] « 
ahifenia@pes Vo tieol « 


svltsnlmess oe thingett “gosth! 
neftanimeece Yo aie e mcite: 
sore yeh te fcomeget © 
ance heges tie “he rnepbal foltetnsmayos = 


—————— nn 


vite: Gea taonregtet gidatetiainy tay)!* 
. shia _ giriaseliales 2289-02 


203 wldadidedte of Dyethe aaneviie na tet) Gievltelonce wvem 6° 
vimon weironire esiupet salieant lovinad toomagenes & alate c 6 
wite wiht eogete qhyalll Segue Mibw episest edi medi mews go 
lo jovhtes ed] Oooeed eletiem beebal | punt gta ali’ 
sip of etoeTin a eh ens res “1 
= 


~ 22 - 
Exhibit 6 
AUDIT OBSERVATION 


No due regard to economy in acquiring the 


C.D. Howe Building. 


Criteria: - As in Exhibit 2. 
- As in Exhibit 3. 


- As in Exhibit 4. 


Condition: - Weak option definition and analysis. 
=: TB process not followed. 
~ No reliable forecasting of inflation. 
“ Substantial design changes are allowed 


after project approval. 


Effects: - Project cost $12 million more than an 
alternative method of acquisition. 
= Cost of project is $102 million compared 


to planned and approved $57 million. 


Causes: = Wish of ADM to contract out rather than 
have DPW construct the building. 
= Inflation. 
- Wish of project architect to make major 
design changes. 


Recommendations: (Specified to eliminate conditions and comply 


with criteria). 
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In addition, a weakness in that particular control in the management 
process will most likely produce more than one adverse effect. Again, to prove 


this requires extensive, costly testing. 


Since the management process consists of many systems, sub-systems 
and controls, a systematic approach should be developed for planning the audit 


tests efficiently. 


Furthermore, the thrust of our audit efforts should be directed 
towards the results and seeking causes of deficiencies in those results. The audit 
is not served by identifying deficiencies in causes, such as deficiencies in some 
control, calling them results and then proceeding to speculate about the causes in 


the deficiencies of causes! 


Recommendations 


As an aid for identifying key controls and their relationships to 
significant effects, the use of a matrix is reeommended. We found that the 
approach described in this paper amplified the audit rationale at the following 


stages of the comprehensive audit process: 


Identifying significant systems and key controls. 
Conducting the preliminary evaluation of systems and key controls. 


Determining causes and effects for unsatisfactory systems. 
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ACCOPRESS' 4 


25070 YELLOW/JAUNE BY2507 
25071 BLACK/NOIR BG2507 
25072 BLUE/BLEU BU2507 
25073 R. BLUE/BLEU R. BB2507 
25074 GREY/GRIS BD2507 
25075 GREEN/VERT BP2507 
25077 TANGERINE BA2507 
25078 RED/ROUGE BF2507 


25079 X. RED/ROUGE X. BX2507 
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